Hybrid dataflow processor

ABSTRACT

A network device that processes a stream of packets has an ingress front end. The ingress front end determines whether the packets are handled in a bounded latency path or in a best-effort path. The bounded latency path packets are granted a resource with a higher priority than the best-effort path packets. As the packets are processed through a number of processing stages, with processing engines, the bounded latency packets are processed within a period of time corresponding to a guaranteed rate. Resources are granted to the best-effort path packets only when the processing engines determine that the resource grant will not impact the latency bounds with respect to the first packets.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority from provisional application Ser.No. 61/645,306, filed on May 10, 2012, and from provisional applicationSer. No. 61/706,513, filed on Sep. 27, 2012. The disclosures of theseprovisional applications are incorporated herein in their entirety byreference.

BACKGROUND

The current disclosure relates to a network device that processespackets.

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

A modern network device that processes packets typically includesprocessing units or “processing engines” serially arranged to processpackets in what is understood as a pipeline configuration. In a pipelineconfiguration, packets are processed in stages so as to achieve a highrate of throughput.

SUMMARY

As an overview, the example embodiments discussed below generally relateto a network device that processes packets through a number ofprocessing units using two paths: a “hard” path and a “soft” path.Packets that traverse the hard path are processed in a manner thatprovides bounded latency. Packets that traverse the soft path areprocessed in a best-effort manner. Notwithstanding the foregoing, notevery example embodiment is required to possess all or even any of thefeatures mentioned in this paragraph.

According to one example embodiment, there is provided a network devicefor processing packets, the device having an ingress front endconfigured to identify a first packet in a stream of packets as a packetto be processed with a guaranteed rate at each stage of a plurality ofprocessing stages, and a second packet in the stream of packets as apacket to be processed without a guaranteed rate at each stage of theplurality of processing stages; a plurality of engines configured toprovide a respective resource for use in processing the stream ofpackets at a processing stage; the processing stages being configuredto: (1) for the first packet, selectively cause a first packetprocessing operation to be performed on the first packet using theresource obtained from one of the engines and to pass the first packetto the next processing stage within a period of time corresponding tothe guaranteed rate; and (2) for the second packet, selectively requestthe resource from one of the engines for processing the second packet,buffer the second packet in a buffer until the resource is available,cause a second packet processing operation to be performed on the secondpacket using the available resource, and to pass the second packet tothe next processing stage in at a rate that is not guaranteed.

In another example embodiment, the processing stages are arranged in apipeline. In another example embodiment, the ingress front end, theengines, and the plurality of processing stages are disposed on a sameintegrated circuit device. The stream of packets includes a plurality ofpackets to be processed with the guaranteed rate of processing, and theone or more engines are configured to grant the resource, to theprocessing stage requesting the resource for the second packet, inresponse to determining that granting the resource to the second packetwill not affect the guaranteed rate of the packets to be processed withthe guaranteed rate. The ingress front end is configured, in an exampleembodiment, to provide data packets as a majority of the packets to beprocessed with the guaranteed rate to the plurality of processingstages.

The network device according to an example embodiment further includes aback pressure bus, wherein a downstream processing stage is configuredto signal to an upstream processing stage, over the back pressure bus,whether the downstream processing stage is available to receive thesecond packet for additional processing. In another example embodiment,the downstream processing stage signals to the upstream processing stagewhether to pass the second packet based on a buffer fill degree of thedownstream processing stage. In response to a buffer of a downstreamprocessing stage reaching a predetermined fullness threshold, thedownstream processing stage sends a signal to an upstream processingstage to stop sending packets. In addition, the network device has softpath buffers disposed in the processing stages, the soft path buffersbeing configured to buffer the second packet for a variable period oftime that is a function of a request to grant access to the resourcefrom a processing stage to one or more of the engines and a grant by oneor more engines to the requesting processing stage to make the resourceavailable for processing the second packet.

In an example embodiment, the plurality of processing stages arearranged in a pipeline, and the soft path buffers are configured toprovide a back pressure signal to a preceding stage indicating whetheror not to send forward in the pipeline another packet to be processedwithout the guaranteed rate. In another example embodiment, the softpath buffer is configured to assert the back pressure signal in responseto one of the soft path buffers exceeding a fullness threshold, and theingress front end is configured to receive the back pressure signal andnot send another packet to be processed without a guaranteed rate untilthe back pressure signal is de-asserted.

In yet another example embodiment, the plurality of processing stagesare arranged to provide: a soft path which processes packets without aguaranteed rate; and a hard path which processes packets with aguaranteed rate irrespective of a processing load exerted on the enginesthe soft path.

In an example embodiment, there is provided a method includingidentifying a first packet in a stream of packets as a packet to beprocessed with a guaranteed rate at each stage of a plurality ofprocessing stages; identifying a second packet in a stream of packets asa packet to be processed without a guaranteed rate at each stage of theplurality of processing stages; requesting at a processing stage aresource from an engine so that an operation can be performed on thefirst packet within a bounded period of time defined by predeterminedlatency corresponding to the guaranteed rate; executing a processingoperation on the first packet using the resource, and passing the firstpacket to a next stage within a period of time defined by the guaranteedrate; requesting permission to request the resource from the engine sothat an operation can be performed on the second packet; in response toa resource request, making a determination as to whether using theresource will affect the guaranteed rate of the first packet, andproviding a resource grant in accordance with the determination; and inresponse to the resource grant, requesting the resource from the engine.

In an example embodiment, the method also includes buffering theresource request until the determination indicates the guaranteed rateof the first packet will not be affected.

In yet another embodiment, a network device is provided for processing aflow of first packets with substantially a guaranteed rate and forsimultaneously processing a flow of second packets with a variable rate.The device also includes one or more engines supplying a processingresource for use in processing the first packets and for use inprocessing the second packets; and a processing stage coupled to the oneor more engines, the processing stage being configured to: process thefirst packets using the resource obtained from the one or more engineswithin a time period defined by the guaranteed rate, and make a resourcerequest for the resource for processing a second packet among the secondpackets, buffer the second packet until one or more of the enginesindicates an availability of the resource for processing the secondpacket without causing the time period for processing the first packetsto exceed the guaranteed rate.

In an example embodiment, there is provided a pipeline processorcomprising a plurality of processing stages including the processingstage, the plurality of processing stages each configured to perform oneor more processing operations on the first packets and the secondpackets. The device determines when to grant the resource based onwhether the engine can grant the resource without affecting theguaranteed rate of the first packets. When there is a plurality ofengines including the engine, each of the engines having a resource andbeing coupled to the processing stage, the processing stage beingconfigured to generate a resource request with respect to more than oneof the plurality of engines. In an example embodiment, the processingstage provides the resource request to each of the plurality of engines.

In an additional example embodiment, a network device also includes apacket classifier, disposed upstream of the processing stage and theengine, configured to classify a packet in a stream of packets as one ofthe first packets or one of the second packets.

DRAWINGS

FIG. 1 is a highly simplified illustrative drawing to provide a highlevel overview of the concept of operation for according to exampleembodiments;

FIG. 2 illustrates a hard real-time system according to exampleembodiments;

FIG. 3 illustrates a soft real-time system according to exampleembodiments;

FIG. 4 illustrates a hybrid system, which has the hard real-time systemof FIG. 2 and the soft real-time system of FIG. 3, according to exampleembodiments; and

FIG. 5 illustrates a flow chart of an example method according toexample embodiments.

DETAILED DESCRIPTION

In the following discussion, descriptions of well-known functions andconstructions may be omitted for increased clarity and conciseness.

Conceptual Overview

FIG. 1 shows a network device 300. A stream of network packets 1 isintroduced. Some packets 1H are to be processed via the hard path, andsome packets 1S are to be processed via the soft path. A component 2 inthe device causes hard path packets 1H to be processed via the hard path3, and causes soft path packets 1S to be processed via the soft path 4.It will be understood that the illustration of two physical paths inFIG. 1 and in other figures is provided for the sake of explanation, andis not to be understood as limiting the example embodiments.

The hard path packets 1H traverse the hard path 3 by passing throughprocessing stages 5. In FIG. 1, only two stages are shown, 5-1 and 5-2.The processing stages are implemented by processing units and involvethe use of various resources (not shown) which will be described in moredetail below. Although the example embodiments do not require a changeto a packet to occur as a consequence of each processing stage, FIG. 1illustrates a change in packets by a change in the label H beforeprocessing stage S1 (i.e., 5-1) to the label H′ afterward. Likewise,FIG. 1 shows the hard path packets that have been through processingstage S2 with the label H″. After the final processing stage S2, thenetwork packets thus processed egress.

The hard path packets 1H that traverse the hard path 3 are processed bythe processing units (not shown) of the processing stages 5 in a mannerthat provides bounded latency. Here, the term “bounded latency” may beunderstood to have a connotation similar to “without delay” or similarto “at a guaranteed throughput rate”.

The soft path packets 1S traverse the soft path 4. The soft path 4 isillustrated, for the sake of explanation only, as a figurative waitingline of soft path packets, waiting to be processed. Soft path packets 1Senter the soft path at the end of the line, and are eventuallyprocessed.

Some concrete example embodiments implementing a soft path are describedbelow, but the overall concept is that the soft path packets 1S areprocessed in a best-effort manner. As shown in FIG. 1, the soft pathpackets 1S advance from a pre-processed state illustrated by the label Sto a post-processed state illustrated by the label S′. While the changefrom S to S′ is depicted simply by a solid line with a solid arrow,positioned between the waiting line and a dashed-line soft path packet1S, it is to be understood that the resources which carry out theprocessing are the same resources that carry out the processing of thehard path packets 1H in the processing stages 5.

That is to say, processing stages 5 involve resources that are intendedto process hard path packets 1H with bounded latency (i.e., withoutdelay), but the idea is that these same resources are to be used toprocess the soft path packets 1S so long as the handling of the hardpath packets 1H is not adversely impacted.

To that end, according to example embodiments, the device 300 is shownwith the soft path 4 issuing requests (“Req” in FIG. 1) to theprocessing stages 5. The concept behind these requests is that they areto be handled at the discretion of the processing stages 5. Below,various other approaches are also discussed, but the idea is that theprocessing units of the processing stages 5 are in the best position todetermine whether their respective processing workload permits thehandling of the processing being requested on behalf of the soft pathpackets 1S.

Once the processing of a given soft path packet is performed, theprocessed packet S′ may egress or be disposed of in any mannerappropriate to the situation.

Looking deeper at the implications of bounded latency for hard pathpackets and best effort processing for soft path packets leads to somepoints to keep in mind as the discussion continues. First of all, delayscannot be tolerated in the hard path, and so it is likely that, fromtime to time, a few packets in the hard path will end up being droppedso that the great majority of packets can traverse the hard path withinthe bounds of permitted latency. On the other hand, since there is noparticular time-wise expectation as to when a soft path packet will beprocessed, it would be relatively rare that a soft path packet wouldneed to be dropped. At the risk of over-generalizing, it may be saidthat (in general) the hard path offers guaranteed timing but notguaranteed delivery, whereas the soft path offers guaranteed deliverybut not guaranteed timing.

With this general framework in mind, the discussion below will focus, inturn, on (1) the terminology used in the remainder of this document, (2)an implementation of a hard path only network device (i.e., a networkdevice with only a hard path), on (3) an implementation of a soft pathonly network device, and on (4) a “hybrid” network device having both ahard path and a soft path. The discussion will also move from theconceptual to the concrete so that a number of instructive, exampleembodiments are presented.

Terminology

This section of the discussion sets forth the technical terms that willbe used to explain the example embodiments below.

Particular embodiments of the present disclosure relate to a networkdevice having network and/or packet processors including a plurality ofprogrammable processing units configured to process a stream of networkpackets. One example of such a processing unit which may beadvantageously employed is a PACKET INSTRUCTION SET COMPUTER (PISC)processor available from MARVELL TECHNOLOGY GROUP, LTD.

The processing units are, in an example embodiment, arranged as apipeline having serially arranged programmable processing stagesconfigured to perform packet specific processing operations on packetsin the stream of packets. One or more engine access points (EAP) areincluded in the pipeline among the programmable processing units and areconfigured to obtain specialized resources from one or more engines,such as lookup tables, accelerators and the like, that are external tothe processing pipeline.

The pipeline interacts with one or more engines via the seriallyarranged EAPs. For example, an EAP can be used to send a request onbehalf of a packet to an engine and to receive a corresponding responsewhich can be integrated into the packet. The request sent on behalf of apacket is used for various packet-processing operations, such asaccessing lookup tables and databases, which are performed using theengine's shared resources and accelerators which are outside thepipeline.

The stream of packets include, in an example embodiment, both hard pathpackets and soft path packets. One example of a packet type that is wellsuited for handling in the hard path is data packets. Data packets willtypically be the most common and the most time-sensitive type of networkpackets, and the rapid movement of data is a worthwhile goal in anynetwork.

One example of a packet type that is sensibly handled in the soft pathis control message (CM) packets. CM packets typically do not need to beprocessed at a guaranteed rate and bounded latency and, therefore, arewell suited to being handled on a best effort basis so as to not disturbthe guaranteed rate and bounded latency of the hard path packets, in anexample embodiment.

Compared to data packets, CM packets typically request access to anengine less frequently, for example because some CM packets perform moretime-consuming table-management operations. Thus, in order to maximizethroughput, data packets are allowed to use as much of the engine'scapacity as possible. Additionally, in some embodiments, because CMpackets are comparatively rare relative to data packets, reserve enginecapacity is not allocated merely to service these packets.

It is easy to understand why data packets are generally well suited tohandling via the hard path, and easy to understand why CM packets aregenerally well suited to handling via the soft path. Therefore, thediscussion below will often use the term “data packets” instead of theterm “hard path packets,” and the term “CM packets” instead of the term“soft path packets.” It is to be understood, however, that these termsare used only to simplify the reader's understanding. There may besituations in which a CM packet should be handled via the hard path, andthere may be situations in which a data packet should be handled via thesoft path. Moreover, there a numerous other types of packets that arenot specifically discussed herein, and there are numerous types ofpackets that may be developed in the future.

Thus, the example embodiments discussed below in terms of “data packets”and “CM packets” should not be interpreted in a limitative sense butonly in an instructive sense.

Hard Real-Time System

A hard real-time system has one or more hard-paths each having a fixedmaximum latency and guaranteed performance. This means that a packetflowing through a hard real-time system will be processed within acertain amount of time that is determined at some stage during systemdesign and/or production. Thus, each of the hard paths has a fixedresource allocation from among the resources of the engines so as toprovide guaranteed performance at a guaranteed rate and bounded latency.

FIG. 2 illustrates a hard real-time system 100 according to exampleembodiments of the present disclosure. The hard real-time system 100includes a pipeline processor 10 disposed in a network processor unit orpacket processor unit that in some embodiments is part of a networkdevice such as a switch. As seen in FIG. 2, the packet processor 10 isin communication with a pipe arbiter 20 and engines 30-1 to 30-n that,in turn, are coupled to packet processor 10 by way of a bus. Thepipeline processor 10 interacts with the engines 30 through engineaccess points (EAPs) 34-1 . . . 34-n located at various stages of thepipeline processor 10. If the EAPs 34-1 . . . 34-n are not distinguishedfrom one another, they are simply referred to as the “EAP 34”, also thenumber of EAPs 34 may be different than the number of the engines 30.The pipeline 10 includes one or more associated processor cores 14-1 . .. 14-n, which if not distinguished from one another are simply referredto as the “processor core 14”. In an embodiment the processing cores areprogrammable cores, such as PISC processing units, that are configuredto perform a programmable context based processing operation on packetsthat flow through the pipeline. The EAPs 34 are a specialized I/O unitfor classification tasks and can each request resources from the engines30 to process a packet and receive a corresponding response which can beintegrated into the packet using the associated processor cores 14-1 . .. 14-n. In some embodiments, the EAP 34 also updates the packet contextby itself without the use of the processor core 14.

More specifically, the EAPs 34 unify access to resources (e.g., look-uptables, control tables, forwarding tables, dedicated accelerators, etc.)stored in embedded or external memory (TCAM, SRAM, DRAM, etc.) of theengines 30 and receive context/code that is used by the associatedprocessing cores 14-1 . . . 14-n to process the packets as the packetsflow through pipeline 10.

In this discussion, if the engines 30-1 to 30-n are not distinguishedfrom one another, they are simply referred to in a general sense as the“engine 30”. The pipe arbiter 20 receives packets, which are mostly datapackets and also include some CM packets such as non-executable controlmessages (nXCM), from a traffic manager 40 that stores the packets in abuffer 44. As used herein, the term traffic manager 40 is a queuingsystem that can be as simple as a first-in first-out (FIFO) buffer to anadvanced external buffering system including queues, a schedulinghierarchy and a buffer having a larger capacity than that of those ofthe pipe arbiter 20 and the pipeline 10. The traffic manager 40 may alsobe referred to herein as an ingress front end.

The pipe arbiter 20 stores the packets in a buffer 26 as they arereceived from the traffic manager, rate-shapes (i.e., pack rate-shapes)and schedules the packets for processing using a packet shaper 24, andfeeds the packets to the pipeline processor 10 where they are processed,in an embodiment. More specifically, in an embodiment, at such time thatcode used to drive the various programmable processing cores is compiledinto executable computer instructions, the rate shaper 24 checks themaximum utilization of the engines 30 by summing the worst case bookingof the engines and ensuring that the worst case booking of an engine isless than its actual capacity. These rate-shaped and scheduled packetsthen traverse through pipeline processor 10 at a guaranteed rate andbounded latency. The rate-shaped and scheduled packets can be dropped inthe hard real-time system in order to provide the guaranteed rate andbounded latency. More specifically, in a normal case when the hard pathhas guaranteed resources, the delivery, throughput rate and boundedlatency of packets flowing through the hard path will be guaranteed.However, If any of the resources is over allocated for the hard path,hard path packets may be dropped so as to achieve guarantees for themajority of the packets. In contrast, if resources are over allocatedfor soft path, the packets will not be dropped. Instead they arebuffered and wait for the resources to become available.

In some example embodiments, the pipeline processor 10 is formed of oneor more Packet Instruction Set Computer (PISC) processor units, each ofthe PISC processor units receives at least partially processed packets,a processing context, and suitable code to perform a next processingoperation. The PISC processor is a processor core specifically designedfor packet processing, in an embodiment, although the present disclosureis not necessarily limited thereto and the pipeline processor 10 is, inan example embodiment, implemented as one or more general CPU orprocessor cores. In the hard real-time system 100 the packets cannot bere-ordered because they travel through the pipeline processor 10 as ifadvancing through a fixed-length first-in-first-out (FIFO) device with aguaranteed rate and bounded latency until they reach a sink 50, which invarious embodiments is another pipeline processor, PISC, EAP, or thelike suitable device. For example, in an embodiment, in each clockcycle, all packets in a hard path of the pipeline processor 10 shift onestage ahead to the next processor core 14 or EAP 34 at which asubsequent processing operation is executed. In this example, aninstruction executed at a processor core 14 is always executed tocompletion within a single clock cycle and every instruction can executeup to a certain number (e.g., four) of operations in parallel. Thepacket then continues to the sink 50.

As mentioned above, however, there are, in an example embodiment,multiple hard paths in the hard real-time system 100. These multiplehard paths process packets in parallel, and it is possible for packetsassociated with one hard path to pass packets associated with anotherhard path within the hard real-time system 100.

Thus, as described above the hard real-time system 100 processes packetsat a guaranteed rate corresponding to, for example, wirespeed,guarantees packet order, has a limited amount of buffers, and assumesworst case latency. It is noted that in an embodiment, only header, orother descriptor data structure representing a packet, is passed throughthe pipeline processor 10. Moreover, on average in a typical mix ofpackets having different processing requirements based on, for example,their respective size or length. By way of example, a packet having asize of 60 bytes has a higher packet rate than a packet having a size of1,500 bytes. These packets may have the same processing requirement,e.g., program, but the frequency of the packets is different. Forexample, if there are many long packets the processing is mostly idle.In various embodiments, because of the need to maintain wirespeedprocessing for some packets, a significant portion of resources providedby the engines 30 may not be used at any given time. As a result, unusedresources are available for processing relatively rare CM packets, andother similar packets that are not time bound.

Soft Real-Time System

A soft real-time system has one or more soft-paths each having anon-guaranteed rate and non-bounded latency, and profiling that is notguaranteed, in an embodiment. This means that a packet flowing through asoft real-time system will be processed at some time, but since there isno guaranteed rate and no bounded latency, the time at which the packetwill be processed is not certain. That is, the packets flowing throughthe soft real-time system are processed on a best-effort basis usingavailable resources without negatively impacting processing forinstance, in the hard path.

FIG. 3 illustrates a soft real-time system 200 according to the exampleembodiments of the present disclosure. The soft real-time system 200includes the pipeline processor 10, the pipe arbiter 20, the engines 30,the traffic manager 40, and the sink 50 described above, thereforeredundant descriptions thereof are omitted.

In general, packets are stored on a first-in first-out (FIFO) buffer ofthe traffic manager until they are ingressed to the pipeline by thetraffic manager 40, and passed from processor to processor. Moreover,when a packet reaches an EAP 34, and a resource from an engine 30 isrequired, a general process ensues. That is, the EAP 34 requests theresource and continues to buffer the packet until the engine signalsavailability. Once the engine signals availability, the EAP 34 can thenobtain the resource from the engine 30.

Since the soft real-time system 200 does not have a guaranteed rate andbounded latency and the packets in the soft real-time system will not bedropped, there could be an occasion in which the soft path buffers 18-1. . . 18-n of the EAPs 34 become full. Thus, while the hard real-timesystem is able to drop packets in order to provide the guaranteed rateand bounded latency, the soft real-time system is not designed to droppackets and will buffer a packet until it is able to be processed. Thesoft path buffers 18-1 . . . 18-n, when not being individually referredto, are simply referred to as the “soft path buffer 18”.

Due to the possibility of the soft path buffers 18 becoming full, insome example embodiments, the EAPs 34 provide back pressure feedback(indicated by the dashed arrows) to previous EAPs 34 and to the pipearbiter 20. Also, since the back pressure signal from the pipelineprocessor 10 causes the pipe arbiter 20 to stop sending packets to thesoft path, the buffer 28 of the pipe arbiter 20 may also become full.Therefore, the pipe arbiter 20 is also able to provide a back pressuresignal to the traffic manager 40 telling the traffic manager 40 to stopsending packets to the soft path. During the time that traffic manager40 is not sending packets to the soft path, the packets accumulate inthe deep soft path buffer 46 of traffic manager 40.

In some example embodiments, the back pressure signal is provided in thesoft real-time system 200 via a separate back pressure bus (not shown).

In some example embodiments, other flow control schemes that that use acredit based system rather than the back pressure are used. These creditbased systems are described later.

In the soft real-time system 200, unlike the above-described hardreal-time system 100 and traditional soft paths, the pipe arbiter 20does not try to predict when there will be gaps that allow the CMpackets to be piggy-backed and go in-band with the data packets. Thatis, since there is no guaranteed rate and bounded latency the pipearbiter 20 does not rate-shape the packets, but rather sends them topipeline 10 for processing in response to receiving signal indicatingthat the pipeline is available to process a next packet.

Although the pipe arbiter 20 does not need to rate-shape and schedulethe packets for its own purposes, it is possible to add rate shapingand/or scheduling when some other consideration requires it. However,the pipe arbiter 20 should schedule the packets, in order to supportmultiple hard path and multiple soft paths. If there the back pressuresignal is de-asserted or no packet in any of the hard paths, then thepacket can be scheduled into the soft path. When there are multiple softpaths, the pipe arbiter 40 schedules packets for the different softpaths by e.g., a round robin scheduling scheme.

The pipe arbiter 40 feeds the packets to the pipeline processor 10 whennot prohibited by a back pressure signal. Further, inasmuch as the softpaths do not have the fixed resource allocation of hard paths, the softpaths use the engine's resources as they become available.

In a case where only the soft paths are used, however, it is notpossible to give any rate/latency guarantees other than statisticalperformance.

Hybrid Device

Therefore, according to an example embodiment there is realized a hybridsystem including both the above-mentioned hard real-time system 100 andthe above-mentioned soft real-time system 200, combined over the sameresources. The hybrid system gives the best of both systems and allowsprograms that need guarantees (e.g., Committed Packet Programs (CPP)programs) to get that by traversing the hard path, while also allowingprograms that do not need guarantees (e.g., Excessive Packet Programs(EPP) programs) to access an engine's 30 resources as they becomeavailable via the soft path. As a result, the hard path can reduce extralatency/buffering due to the occasional costly CM packets and the softpath has a much simpler regulation mechanism for CM packets.

Hybrid System

FIG. 4 illustrates an example hybrid system 300. The engines 30 of thehybrid system 300 have separate (logical) buffering systems (queues) forthe hard real-time system 100 and for the soft real-time system 200. Forexample, the engine 30-1 includes a local hard-time buffer 32-1H and alocal soft-time buffer 32-1S, and each of the engines 30 includes arespective local hard-time buffer and a respective local soft-timebuffer.

In the hybrid system, the hard real-time system 100 has priority overthe soft real-time system 200. This is because the hard real-time system100 has a guaranteed rate and bounded latency whereas the soft real-timesystem 200 does not have a guaranteed rate and bounded latency.

Mechanisms for Path Selection

The traffic manager 40 directs the packets that do not need a guaranteedrate and a bounded latency to the soft path buffer 28 in the pipearbiter 20. These packets are mainly CM packets, however, thisdisclosure is not limited thereto and the soft path buffer 28, in anexample embodiment, receives data packets as well. Similarly, thetraffic manager 40 directs the packets that do need guarantees to thehard path buffer 26 of the pipe arbiter 20. These packets are mainlydata packets, however, this disclosure is not limited thereto and thehard path buffer 26, in an example embodiment, receives CM packets aswell.

In an example embodiment, the traffic manager 40 is an advanced buffersystem with multiple queues mapped to an egress port. All the packetsstored in the queues mapped to the egress port will be processed in asame soft path or a same hard path. That is, there is mapping between anegress port of the traffic manager 40 and an input of the soft and hardpaths. In some embodiments, the traffic manager 40 receives packets fromvia an ingress port, stores the packets in its buffer system, andtransmits the packets to the pipe arbiter 30. The traffic manager 40 isnot required to do any processing selection because the processingselection has already been done either by static mapping from oneingress port to a queue, or by a pre-classifier that by a lookup (ine.g. TCAM) on the packet selects one queue in the traffic manner 40.

The traffic manager 40 only selects (schedules) which packets in itsbuffer system/queues to send to the egress port without respect to thetype of processing required by the packets.

In another example embodiment, the traffic manager 40 includes afirst-in first-out (FIFO) buffer for each of the hard paths and softpaths.

The traffic manager 40, in an example embodiment, directs the packets tothe respective hard and soft paths using various techniques such asstatic choice, dynamic choice by service class, and dynamic choice bytraffic frequency, each of which is described below.

In the static choice technique, according to an example embodiment, uponarrival of a packet at the traffic manager 40 it is determined whetherthe packet needs to be sent to the hard path or to the soft path.Specifically, candidates for the hard path include those needingguaranteed wire speed performance or that need to be performed within aspecified time. Examples of these candidates include data traffic,regular OAM ping programs, and some management packets for executingprograms such as a table scrub or programs for detecting timeouts.

Candidates for the soft path include those that simply need “besteffort” performance. Examples of these candidates include managementpackets such as packets that update table entries or packets that readcounters.

In the dynamic choice by service class, according to an exampleembodiment, there is a classification using the pipeline processor 10.In the first pass of a packet through the pipeline processor 10, datatraffic is used for classification of the packet, e.g., hard path forwirespeed. The packet is passed through another pipeline processor 50 orlooped through the same pipeline processor 10 and enter the hard path orthe soft path depending on the earlier classification. Also a set ofTCAM or SIP (e.g., hard coded switch) like pipeline can perform theclassification.

Candidates for the hard path in the second pass include packets withrate-sensitive and latency-sensitive service classes. Examples includesystem control packets, VoIP, gaming, conferencing, etc. Candidates forthe soft path in the second pass include packets that are neitherrate-sensitive nor latency-sensitive service classes. Examples includeVoD, data traffic, scavenger class packets, etc.

In the dynamic choice by traffic frequency, according to an exampleembodiment, there is a classification similar to that described abovewith respect to the dynamic choice by service class technique.Candidates for the hard path in the second pass include bulk traffic,e.g., Layer 2 (L2) Ethernet traffic. Candidates for the soft path in thesecond pass include relatively rare types of traffic that should bemanaged but that one may not want to provide with optimized resources,e.g., Layer 3 (L3) Ethernet traffic.

Mechanism for Requesting Resources

In the hard path, when the packets arrive at the buffers 16 of the EAPs,the EAPs simply request resources on behalf of the packets. Theserequests are stored in the engine's hard path buffer 32-H and they areexecuted in the order they are received by the engines 30.

In contrast, when the back pressure signal is not present, e.g., whenthe back pressure signal is de-asserted, the soft path buffer 28 pushespackets stored thereon to the pipeline processor 10. Upon arriving atthe pipeline processor 10 these packets are stored in the soft pathbuffer 18 of the EAP 34. In response to receiving a packet in the softpath buffer 18, the EAP 34 sends a “request to grant” to one or more ofthe engines 30. In response to receiving the request to grant, theengines 30 determine whether or not they have available resources thatmay be used without disrupting the flow of packets in the hard path. Inresponse to determining that there are available resources that may beutilized without disrupting the flow of packets in the hard path, anengine 30 sends a “grant” to the EAP 34. In response to receiving thegrant from the engine 30, the EAP 34 then requests a resource from theengine that sent the grant and the request is stored in the soft pathbuffer 32-S of the engine and executed in the order that it was receivedby the engine 30. The “request to grant” may be thought of as a resourcerequest, and the “grant” may be thought of as a resource grant.

The engines 30 determine whether or not they have available resourcesbased on its current congestion status, by for example, at least one ofthe buffer fill degree, semaphores, outstanding credits, other localresources, etc., and can therefore make a correct decision quickly as towhether or not they have available resources. Thus there is no need totry to predict the engine congestion from the pipe arbiter 20. As aresult, the non-used bandwidth from all EAPs 34 can be available for thesoft-path.

A request to grant received from the soft path is enqueued in separatequeue(s) at the engine. As described above, the engines each include ahard-path buffer 32-H and a soft-path buffer 32-S. In other words, thesoft-path has a queue structure parallel to that of the hard path, in anembodiment. This is so that a request to consume the engine's 30resources can be held pending, for the soft path, without disturbing thepacket flow in the hard-path.

Since the engines 30 are distributing their resources to the EAPs 34they each act as a broker for the EAPs 34. The engines 30 can regulatethe distribution of their resources in several different ways.

In one example embodiment, the engines 30 regulate the distribution oftheir resources by providing a back pressure signal to each of the EAPs34 via a separate back pressure bus (not shown). Thus, in thisembodiment, the engines 30 need a queue per EAP or the acceptance ofhead-of-line blocking. Therefore, the potential exists for the queues tobecome large due to unstoppable data, and the rate depends on theround-trip. That is to say, the engine 30 achieves a higher buffer fill,e.g., so that multiple EAPs 34 may send simultaneous requests during thesame clock cycle. If the transport latency between the EAP 34 and theengine 30 is more than one clock cycle, then more than one request tothe engine 34 to the engine can be sent before the back pressure signalcan asserted so as to stop the EAP from sending more packets, e.g.,“unstoppable data”.

In another example embodiment, the engines 30 regulate the distributionof their resources via credits. The credits correspond to a bufferposition at the engine 30 and the rate depends on the amount of creditsthe EAP 34 is given. For example, an EAP 34 requests for one or morecredits, then when the engine can grant a credit the requesting EAP 34it does so. Next, when the EAP 34 receives the credit it sends a requestfor the resource to the engine 30. The engine returns credits to the EAP34 when consumed. This can be achieved by piggy backing the credit withthe response or via a separate bus.

In one example embodiment, the engine 30 grants credits to the EAP(s) 34that requested a credit, if there is buffer space in the soft buffer32-S of the engine 30. A credit corresponds to an entry in the softrequest buffer. The engine 30, in one example the engine, has a counter(not shown) for each EAP 34 (or similar client) in order to keep trackof the number of credits currently needed by each EAP 34. The EAP 34 mayrequest a credit from an engine 30 by a signal on a separate bus(increasing a count of the associated counter by, for example, +1). Theengine 30 may also have a broker (not shown) that selects which EAP 34to grant a credit to in a round robin among all EAP 34 that requested acredit. The present disclosure is not limited to a round robin schemeand can also include a weighted round robin scheme, a most outstandingcredits first scheme, e.g., the largest queue first, a strict priorityscheme, etc.

When an EAP 34 is selected its request counter is decreased by, forexample, −1. The engine 30 may grant multiple credits to the same EAP 34or different EAPs 34 in order to get higher throughput. When the enginerequests are consumed and processed by the engine 30, it frees up thesoft buffer 32-1 thereby making the credits available again to bedistributed by the broker. The engine 30 sends credit grants with asignal on a separate bus to the EAP 34. A counter increases by, forexample, +1, which means that the EAP 34 is permitted to request thatnumber of engine resources from the engine and when the request isactually transmitted to the engine 30 the counter is decreased by sameamount, for example, −1.

For example, the broker keeps track of the available credits. The brokerstarts by having X credits corresponding to the number of entries in thesoft buffer and when the broker grants credits to the EAPs 34, theavailable credits are decreased. The broker does not grant credits suchthat the available credits become negative. When an engine requeststored in the engine's soft request buffer 32-S, is consumed by theengine 30 it frees one entry in the soft request buffer 32-S. Then anumber of the broker's available credits increases, and the credit isavailable to be distributed by the broker to one of the EAPs 34 again.

In another example embodiment, the engines 30 regulate the distributionof their resources via a request grant scheme using a separaterequest/grant bus (not shown). For example, an EAP requests a number ofcredits and the engine grants a number of credits to the EAP if it canaccept the requests without disturbing the stream of packets on thehard-path. In this example embodiment, the rate depends on round-triptime, which can be improved by granting multiple credits at a time.

In these embodiments, if the credits are distributed according to thecurrent need, then the engine's 30 soft-path buffer 32-S and enginecapacity could be utilized more efficiently when compared to the relatedart. As described, the engines 30 according to the example embodimentscan act like a broker and distribute their respective resource capacityaccording to the need.

In addition, in some example embodiments, the engines 30 regulate therate by adapting the amount of credits granted to each of the EAPs 34.In these embodiments, the adaptive scheme is based on a particular sendfrequency of an EAP 34. For example, by sending a periodic credit updatevia a credit bus (not shown) where credits that are not used are lostwhen a new update arrives or by sending periodic adaptions {+,−,0}.

Furthermore, a specialized request-grant engine is used, in an exampleembodiment, to implement the broker by, for example, granting requestsin a round-robin type fashion among all EAPs 34 currently requesting agrant to request.

In an alternative embodiment, there is provided a dynamic grant requestscheme. In this embodiment, if an EAP 34 needs to send N requests onbehalf of packets in the soft path to the engine 30, the EAP willrequest N grants to requests from the engine 30. The engine 30 will thensend up to N grants to request to the EAP 34 based on its currentlyprocessing load. In response to receiving the up to N grants to request,the EAP transmits that many requests to the engine 30 where they arestored in the soft path buffer 32-S until they are able to be executedwithout disturbing the flow of packets in the hard path.

To better understand this dynamic grant request scheme assume that anEAP 34 can only request 1 credit at a time, for simplicity ofexplanation. Thus, the EAP 34 needs to use the credit before it requestsa new one.

The EAP 34, in an example embodiment, keeps track of the states perengine based on the granted requests (N). When all granted requests aresent to the EAP 34, then the EAP requests a new credit. In a case wherethere are multiple channels, the EAP 34 submits a request to grant tomultiple engines 30 on behalf of a packet in the soft path. In this casethe EAP either needs to obtain a grant from all the engines 30 to whichthe request to grant was submitted before sending the request, or theEAP 34 sends the request as soon as it is granted and then waits on allthe responses before proceeding to the next stage of the pipelineprocessor 10. Waiting on all responses before proceeding to the nextstage of the pipeline processor 10 makes better use of resources byreducing a time that credits are occupied to be as small as possible. Ina case where there are multiple engines 30, if there is only one requestat a time there is only one state per channel and the simple solution isto only grant one credit at a time. This is because there is no need tokeep track of a later packets pending engine usage. Since the EAP inthis dynamic grant request scheme cannot request credits withoutactually having something to send to the engines 30 so as to avoidlosing credits, this scheme does not require supporting separate Nstates per engine 30.

To better understand the dynamic grant request scheme from the viewpointof the engines 30, assume that there are multiple outstanding credits toeach engine 30, thus overcoming the channel bus latency and thereby onlybeing limited by the request-grant latency of the soft path. Regardingthe states per engine, if M is the current number of available creditswhich are free for filling up the soft path buffer 32-S and N is thenumber of current outstanding granted credits, then when a request isconsumed by the engine 30 M increases by 1. Similarly, when a request isreceived from an EAP 34 then M is decreased by 1. When a credit isgranted by the engine 30 then N increases by 1 and when a request isreceived by the engine 30 from the EAP then N decreases by 1.

Based on the above, M should be greater than or equal to N at all timesin this scheme, except when M less than 0, which means there is a bufferoverflow, and when N is less than 0, which means that there has been anunexpected request.

When an EAP 34 wants to be part of the request scheduling, then it setsa pending request flag. To understand the states per EAP 34 in thisscheme assume that G is an outstanding granted request. G is increasedby 1 if a request is granted. G is decreases by 1 if a request isreceived. N is equal to G except when G is less than 0 meaning thatthere has been an unexpected request.

The engines, in one example embodiment, use time-out (T) to protectagainst lost requests. T is MAX when a new request is granted. The valueof MAX is arbitrarily set so as to be certainly an error but not solarge that instability would occur. T is reduced by, for example, 1 eachclock cycle.

When M is greater than 0, then the engine can grant requests and, tofulfill requests, a round robin between the EAPs 34 with pendingrequests is implemented according to an example embodiment.

In another example embodiment, there is a static credit-based soft path.In this embodiment, the soft path relies on a static credit system wherecredits are distributed at compile time. For the engine 30, B is equalto the total number of credits corresponding to the soft-path buffer32-S. At compile time the credits C are distributed among the EAPs suchthat the sum of C is less than B thereby preventing the possibility thathe soft path will interfere with the packet flow in the hard path. Inone exemplary embodiment, an EAP is assigned more than one credit at atime to get high through-put and thereby overcoming channel bus latency.

Multiple Flows

As described, the hard path, by itself, works like a conventionalprocessing pipe and thus the hard real-time system 100 is not requiredto have more than one hard path. However, it is understood that thisdisclosure is not limited thereto and that, in example embodiments,there is more than one hard path, as indicated by the multiple linesflowing from the traffic manager 40 to the sink through the hardreal-time system 100 in FIGS. 1 and 3.

The soft-path, on the other hand, benefits from different flows, e.g.,for different applications, that can process independently from oneanother e.g., if they use different engines 30. That is, according toexample embodiments, there is more than one soft path, as indicated bythe multiple lines flowing from the traffic manager 40 to the sink 50through the soft real-time system 200 in FIG. 3.

The hybrid systems 300 described herein are, in example embodiments,embodied as a system on a chip (SOC) where each of the above describedcomponents is provided on the same chip. The hybrid systems, in exampleembodiments, are embodied as a SOC where at least the pipeline processor10 and engines 30 are provided on the same chip and other componentsthat are not provided on the same chip are provided as discretecomponents, as other systems on chips, or any combination thereof. Thesehybrid systems 300, according to example embodiments, are embodied asentirely discrete components that are connected via an internal network(bus structure/system bus), or the like.

FIG. 5 illustrates flow chart of an example method according to exampleembodiments described herein. At S100 the pipe arbiter 20 receives apacket and at S105 it is determined whether or not the packet is to beprocessed in the hard path or not.

At S110, in response to determining that the packet is to be processedin the hard path, the pipe arbiter 20 sends the packet to the hard pathof the pipeline 10. At S115, in response to the packet arriving at thehard path of the pipeline 10, the EAP 34 requests a resource from theengine 30 in order to process the packet. At S120, in response toreceiving the resource from the engine 30, the EAP processes the packetwith use of the processor 14. At step S125, if there is anotherprocessing stage in the pipeline 10, then the process repeats beginningwith S115. Otherwise, if there is no next stage, then the processedpacket is sent to the sink 50 and the process ends for that pipeline 10.

At S105, in response to determining that the packet is not to beprocessed in the hard path (e.g., it is destined for the soft path), thepipe arbiter 20 sends the packet to the soft path of the pipeline 10. AtS130, in response to determining that the packet is not to be processedin the soft path, the network device buffers the packet in the pipearbiter 20 unless the back pressure signal is present or asserted. Thatis, in the absence of back pressure, i.e., when the back pressure isde-asserted, the packet is buffered in the pipe arbiter 20. At S135, inresponse to determining that the back pressure signal is present orasserted, the soft real-time system 200 signals upstream to request thatthe traffic manager 40 stop sending packets to be processed in a softpath. This back pressure signal remains asserted until the system backpressure allows the packet to be sent to the pipeline 10 to be stored inthe soft path buffer 18.

At S140, in response to receiving the packet on the soft path, thepacket is buffered in buffer 18. At S145, in response to storing thepacket in the buffer 18, the EAP 34 sends a request to grant to theengine 30.

At S150, in response to receiving a request to grant from the EAP, theengine 30 determines whether or not there are sufficient resourcesavailable to process the packet without affecting the latency of thehard path. In response to determining that there are not enoughresources available to process the packet without affecting the latencyof the hard path, the request to grant is buffered in the soft pathbuffer 32-S of the engine 30 until the request can be granted. If thesoft path buffer 32-S exceeds a predetermined fill level threshold, thenit provides or asserts a back pressure signal to the soft real-timesystem to prevent more packets from being sent to the soft path untilthe soft path buffer's 32-S fill level falls below the predeterminedfill threshold.

At S155, in response to determining that there are enough resourcesavailable to process the packet without affecting the latency of thehard path, the engine sends a grant to the EAP 34. At S160, in responseto receiving the grant, the EAP requests the resource from the engine 30in order to process the packet.

At S165, in response to processing the packet, it is determined whetheror not there is another stage in the pipeline 10. If there is notanother stage, then the process for that packet in this pipeline 10ends. Otherwise, if there is another stage, then at S170, the softreal-time system determines whether back pressure signal is de-assertedso as to allow the packet to be buffered in the next stage.

At 175, in response to determining that the back pressure is asserted soas to not allow the packet to be buffered in the next stage, the softreal-time system 200 signals the upstream stages to not send anotherpacket and the soft real-time systems continues to monitor the backpressure signal until the back pressure signal is de-asserted so as toallow the packet to be sent to the next stage. At 180, in response todetermining that the back pressure signal is de-asserted so as to allowthe packet to be buffered in the next stage, the soft real-time system200 signals the upstream stages to send another packet and the packet isstored in the buffer 18 of the next stage thereby looping the processback to S140.

Conclusion

Although the inventive concept has been described above with respect tothe various embodiments, it is noted that there can be a variety ofpermutations and modifications of the described features by those whoare familiar with this field, without departing from the technical ideasand scope of the features, which shall be defined by the appendedclaims.

Further, while this specification contains many features, the featuresshould not be construed as limitations on the scope of the disclosure orthe appended claims. Certain features described in the context ofseparate embodiments can also be implemented in combination. Conversely,various features described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination.

Although the drawings describe operations in a specific order and/orshow specific arrangements of components, and are described in thecontext of access segments of data centers, one should not interpretthat such specific order and/or arrangements are limited, or that allthe operations performed and the components disclosed are needed toobtain a desired result. There are numerous hardware and softwaredevices that can be configured to forward packets, transmit variousaddress resolution messages, and update address caches and packetaddresses in the manner described in the present disclosure with respectto various embodiments. Accordingly, other implementations are withinthe scope of the following claims.

There is claimed:
 1. A network device for processing packets, thenetwork device comprising: an ingress front end configured to identify(i) a first packet in a stream of packets as a packet to be processedwith a guaranteed rate at each processing stage of a plurality ofprocessing stages that are programmable, and (ii) a second packet in thestream of packets as a packet to be processed without a guaranteed rateat each processing stage of the plurality of processing stages; aplurality of engines configured to provide a respective resource for usein processing the stream of packets at a processing stage, the resourcebeing external to the plurality of processing stages; the plurality ofprocessing stages are arranged in a pipeline and configured to: (i) forthe first packet, selectively cause a first packet processing operationto be performed on the first packet using the resource obtained from oneof the engines and to pass the first packet to the next processing stagewithin a period of time corresponding to the guaranteed rate; and (ii)for the second packet, selectively request the resource from one of theengines for processing the second packet, buffer the second packet in abuffer until the resource is available, cause a second packet processingoperation to be performed on the second packet using the availableresource, and, in response to receiving a backpressure signal fromdownstream in the pipeline indicating an ability to forward the secondpacket to the next processing stage in the pipeline, to pass the secondpacket to the next processing stage at a rate that is not guaranteed. 2.The network device for processing packets according to claim 1, whereinthe ingress front end, the engines, and the plurality of processingstages are disposed on a same integrated circuit device.
 3. The networkdevice for processing packets according to claim 1, wherein the streamof packets includes a plurality of packets to be processed with theguaranteed rate of processing, and the one or more engines areconfigured to grant the resource, to the processing stage requesting theresource for the second packet, in response to determining that grantingthe resource to the second packet will not affect the guaranteed rate ofthe packets to be processed with the guaranteed rate.
 4. The networkdevice for processing packets according to claim 3, wherein the ingressfront end is configured to provide data packets as a majority of thepackets to be processed with the guaranteed rate to the plurality ofprocessing stages.
 5. The network device according to claim 1, furthercomprising a back pressure bus, wherein a downstream processing stage isconfigured to signal to an upstream processing stage, over the backpressure bus, whether the downstream processing stage is available toreceive the second packet for additional processing.
 6. The networkdevice according to claim 5, wherein the downstream processing stagesignals to the upstream processing stage whether to pass the secondpacket based on a buffer fill degree of the downstream processing stage.7. The network device according to claim 1, wherein, in response to abuffer of a downstream processing stage reaching a predeterminedfullness threshold, the downstream processing stage sends a signal to anupstream processing stage to stop sending packets.
 8. The network deviceaccording to claim 1, further comprising soft path buffers disposed inthe processing stages, the soft path buffers being configured to bufferthe second packet for a variable period of time that is a function of arequest to grant access to the resource from a processing stage to oneor more of the engines and a grant by one or more engines to therequesting processing stage to make the resource available forprocessing the second packet.
 9. The network device according to claim8, wherein the soft path buffers are configured to provide the backpressure signal to a preceding processing stage indicating whether ornot to send forward in the pipeline another packet to be processedwithout the guaranteed rate.
 10. The network device according to claim8, wherein, the soft path buffer is configured to assert the backpressure signal in response to one of the soft path buffers exceeding afullness threshold, and wherein the ingress front end is configured toreceive the back pressure signal and not send another packet to beprocessed without a guaranteed rate until the back pressure signal isde-asserted.
 11. The network device according to claim 1, wherein theplurality of processing stages are arranged to provide: a soft pathwhich processes packets without a guaranteed rate; and a hard path whichprocesses packets with a guaranteed rate irrespective of a processingload exerted on the engines the soft path.
 12. A method comprising:identifying a first packet in a stream of packets as a packet to beprocessed with a guaranteed rate at each processing stage of a pluralityof processing stages that are programmable and arranged in a pipeline;identifying a second packet in a stream of packets as a packet to beprocessed without a guaranteed rate at each processing stage of theplurality of processing stages; requesting at a processing stage aresource from an engine so that an operation can be performed on thefirst packet within a bounded period of time defined by predeterminedlatency corresponding to the guaranteed rate, the resource beingexternal to the plurality of processing stages; executing a processingoperation on the first packet using the resource, and passing the firstpacket to a next processing stage within a period of time defined by theguaranteed rate; requesting permission to request the resource from theengine so that an operation can be performed on the second packet; inresponse to a resource request, making a determination as to whetherusing the resource will affect the guaranteed rate of the first packetaccording to a backpressure signal from downstream in the pipelineindicating whether or not to send forward in the pipeline another packetto be processed without the guaranteed rate, and providing a resourcegrant in accordance with the determination; and in response to theresource grant, requesting the resource from the engine.
 13. The methodof claim 12, further comprising buffering the resource request until thedetermination indicates the guaranteed rate of the first packet will notbe affected.
 14. A network device for processing a flow of first packetswith substantially a guaranteed rate and for simultaneously processing aflow of second packets with a variable rate, the network devicecomprising: one or more engines supplying a processing resource for usein processing the first packets and for use in processing the secondpackets; and a plurality of processing stages arranged in a pipeline andcoupled to the one or more engines and external to the resource, theplurality of processing stages being programmable and configured to:process the first packets using the resource obtained from the one ormore engines within a time period defined by the guaranteed rate, make aresource request for the resource for processing a second packet amongthe second packets, buffer the second packet until one or more of theengines indicates an availability of the resource for processing thesecond packet without causing the time period for processing the firstpackets to exceed the guaranteed rate, and in response to receiving abackpressure signal from downstream in the pipeline indicating anability to forward the second packet to the next processing stage in thepipeline, to pass the second packet to the next processing stage at arate that is not guaranteed.
 15. The network device according to claim14, wherein the plurality of programmable processing stages are eachconfigured to perform one or more processing operations on the firstpackets and the second packets.
 16. The network device according toclaim 15, wherein the engine determines when to grant the resource basedon whether the engine can grant the resource without affecting theguaranteed rate of the first packets.
 17. The network device accordingto claim 14, further comprising a plurality of engines including theengine, each of the engines having a resource and being coupled to theprocessing stage, the processing stages being configured generate aresource request with respect to more than one of the plurality ofengines.
 18. The network device according to claim 17, wherein theprocessing stages provide the resource request to each of the pluralityof engines.
 19. The network device according to claim 14, furthercomprising a packet classifier, disposed upstream of the processingstage and the engine, configured to classify a packet in a stream ofpackets as one of the first packets or one of the second packets.